Systems and methods for intelligently configuring and deploying a control structure of a machine learning-based dialogue system

ABSTRACT

A system and method of configuring a graphical control structure for controlling a machine learning-based automated dialogue system includes configuring a root dialogue classification node that performs a dialogue intent classification task for utterance data input; configuring a plurality of distinct dialogue state classification nodes that are arranged downstream of the root dialogue classification node; configuring a graphical edge connection between the root dialogue classification node and the plurality of distinct state dialogue classification nodes that graphically connects each of the plurality of distinct state dialogue classification nodes to the root dialogue classification node, wherein (i) the root dialogue classification node, (ii) the plurality of distinct classification nodes, (iii) and the transition edge connections define a graphical dialogue system control structure that governs an active dialogue between a user and the machine learning-based automated dialogue system.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.62/779,945, filed 14 Dec. 2018, which is incorporated in its entirety bythis reference.

GOVERNMENT RIGHTS

The subject matter of the invention may be subject to U.S. GovernmentRights under National Science Foundation grants: NSF SBIR Phase 1Grant-1622049 and NSF SBIR Phase 2 Grant-1738441.

TECHNICAL FIELD

The inventions herein relate generally to the machine learning andartificially intelligent dialogue systems fields, and more specificallyto a new and useful system and method for intelligently controlling amachine learning-based conversational service in the machine learningfield.

BACKGROUND

Modern virtual assistants and/or online chatbots may typically beemployed to perform various tasks or services based on an interactionwith a user. Typically, a user interacting with a virtual assistant maypose a question or otherwise submit a command to the virtual assistantto which the virtual assistant may provide a response or a result. Manyof these virtual assistants may be implemented using a rules-basedapproach, which typically requires coding or preprogramming many orhundreds of rules that may govern a manner in which the virtualassistant should operate to respond to a given query or command from auser.

While the rules-based approach for implementing a virtual assistant maybe useful for addressing pointed or specific queries or commands made bya user, the rigid or finite nature of this approach severely limits acapability of a virtual assistant to address queries or commands from auser that exceed the scope of the finite realm of pointed and/orspecific queries or commands that are addressable by the finite set ofrules that drive the response operations of the virtual assistant.

That is, the modern virtual assistants implemented via a rules-basedapproach for generating responses to users may not fully satisfy queriesand commands posed by a user for which there are no predetermined rulesto provide a meaningful response or result to the user.

Additionally, while machine learning enhances capabilities ofartificially intelligent conversational systems, inefficiencies continueto persist in the underlying control structures used for controlling theunderlying machine learning models performing classification andpredictive functions of the artificially intelligent conversationsystems.

Therefore, there is a need in the machine learning field for systems andmethods that enable a configuring of a dynamic dialogue system controlstructure that is capable of evolving to handle simple or complexconversations between a virtual dialogue agent and a user. Theembodiments of the present application described herein providetechnical solutions that address, at least, the need described above, aswell as the deficiencies of the state of the art described throughoutthe present application.

BRIEF SUMMARY OF THE INVENTION(S)

In one embodiment, a method of configuring a graphical control structurefor controlling a machine learning-based automated dialogue systemincludes configuring a root dialogue classification node that performs adialogue intent classification task for utterance data input to amachine learning-based automated dialogue system; configuring aplurality of distinct dialogue state classification nodes that arearranged downstream of the root dialogue classification node, whereineach of the plurality of distinct dialogue state classification nodesperforms a distinct dialogue classification task for a distinct dialogueintent based on the utterance data; configuring a graphical edgeconnection between the root dialogue classification node and each of theplurality of distinct state dialogue classification nodes thatgraphically connects each of the plurality of distinct state dialogueclassification nodes to the root dialogue classification node therebyenabling a dialogue path from the root node downstream to each of theplurality of distinct state dialogue classification nodes, wherein (i)the root dialogue classification node, (ii) the plurality of distinctclassification nodes, (iii) and the transition edge connection betweenthe root dialogue classification node and each of the plurality ofdistinct dialogue classification nodes define a graphical dialoguesystem control structure that governs an active dialogue between a userand the machine learning-based automated dialogue system.

In one embodiment, the root dialogue classification node comprises asingle machine learning model that is trained to perform multipledistinct dialogue intent classification tasks that produces an intentclassification prediction based on the input of the utterance data thataligns with a distinct dialogue intent competency of one of theplurality of distinct dialogue classification nodes downstream of theroot dialogue classification node.

In one embodiment, the method includes traversing the dialogue betweenthe user and the machine learning-based automated dialogue system fromthe root dialogue classification node downstream to a distinct one ofthe plurality of distinct dialogue classification nodes based on theintent classification prediction.

In one embodiment, the root dialogue classification node comprises anensemble of distinct machine learning models, each of the distinctmachine learning models being trained to perform a distinct dialogueintent classification tasks, wherein the ensemble produces multipledistinct intent classification predictions including a distinct intentclassification prediction for each one of the plurality of distinctdialogue classification nodes downstream of the root dialogueclassification node.

In one embodiment, the method includes traversing the dialogue betweenthe user and the machine learning-based automated dialogue system fromthe root dialogue classification node downstream to a distinct one ofthe plurality of distinct dialogue classification nodes based onidentifying which distinct intent classification prediction as having ahighest level of confidence.

In one embodiment, the method includes re-classifying, by the rootdialogue classification node, the active dialogue between the user andthe machine learning-based automated dialogue system based on areversion of the active dialogue from one state dialogue classificationnode of the plurality of distinct state dialogue classification nodesarranged downstream of the root dialogue classification node.

In one embodiment, the root dialogue classification node re-classifiesthe active dialogue from the one state dialogue classification node to asecond distinct state dialogue classification node of the plurality ofdistinct state dialogue classification node. of the dialogue intentcompetency of the reverting state node.

In one embodiment, the one state dialogue classification node revertsthe active dialogue to the root classification node based on computing amisalignment between dialogue intent capabilities of the one statedialogue classification node and an identified current dialogue intentof one or more utterances from the user.

In one embodiment, the method includes a plurality of distinct dialoguecompetency sub-networks, wherein each of the plurality of distinctdialogue competency sub-networks: separately extends from the rootdialogue classification node via one graphical edge connection, includesa progenitor node comprising a distinct one of the plurality of distinctstate dialogue classification nodes, wherein one or more sub-dialogueintent nodes are arranged downstream and extend from the progenitornode.

In one embodiment, the each of the plurality of distinct dialoguecompetency sub-networks have a hierarchical structure in which: theprogenitor node defining a top layer of the hierarchical structure isconfigured with coarse capabilities to perform dialogue intentclassification tasks for generating predictions identifying one or moremultiple distinct sub-intents of a single dialogue competency or singledialogue category, the one or more sub-dialogue intent nodes definingone or more subsequent layers from the progenitor node is configuredwith granular capabilities to perform dialogue intent classification forat least a distinct one of the multiple distinct sub-intents.

In one embodiment, the method includes if a misalignment between acomputed dialogue intent of an utterance of a user and a dialogueclassification capability of a sub-dialogue intent classification nodeof a first distinct sub-network of the plurality of distinct dialoguecompetency sub-networks is detected, configuring the machinelearning-based dialogue system to pass, in a jumping manner without agraphical edge connection, the active dialogue from the first distinctsub-network to a second distinct sub-network having the dialogueclassification capability matching the dialogue intent of the user.

In one embodiment, the method includes configuring a classificationtransition edge connection, wherein the classification transition edgeconnection relates to a graphical transition connection between at leasta pair of distinct state classification nodes indicating that a machinelearning classification prediction has been generated by an originatingnode of the pair enabling an active dialogue between the user and themachine learning-based dialogue system to pass from the originating nodeto a destination node of the pair.

In one embodiment, the method includes configuring a slot transitionedge connection, wherein the slot transition edge connection relates toa graphical transition connection between at least a pair of distinctstate classification nodes indicating that one or more slotclassification values of an active dialogue between the user and themachine learning-based dialogue system which are passed pass from theoriginating node to a destination node of the pair.

In one embodiment, the method includes configuring an update transitionloop connection, wherein the update transition loop connection relatesto a graphical transition connection that graphically originates at adistinct state classification node and that loops back into the distinctstate classification node indicating an action to be performed by thedistinct state classification node for obtaining additional informationbeyond an initial utterance from the user to the distinct stateclassification node.

In one embodiment, the method includes configuring a logic transitionedge connection, wherein the logic transition edge connection relates toa graphical logical transition connection between at least a pair ofdistinct state classification nodes indicating that enables an executionor a completion of one or more business tasks for generating a responseto dialogue input from the user.

In one embodiment, the method includes setting an archetype that isselected from a plurality of distinct archetypes for each of the one ormore new dialogue competencies of the target dialogue system based onattributes of mishandled utterance data within a target corpus ofmishandled utterances of the one or more distinct corpora of mishandledutterances.

In one embodiment, the method includes deploying the graphical dialoguesystem control structure to control an operation of each of the rootdialogue classification nodes and each of the plurality of distinctstate dialogue classification nodes for implementing an artificiallyintelligent dialogue agent that interfaces with the user in a livedialogue.

In one embodiment, the dialogue system control structure relates to agraphically based structure that governs an operation of andcommunication between multiple machine learning models and dialogueresponse generating components of the machine learning-based dialoguesystem.

In one embodiment, a system for intelligently configuring a graphicalcontrol structure for controlling a machine learning-based automateddialogue system, the system comprising: a web-based user interface forconfiguring a graphical dialogue system control structure that is inoperable communication a remote machine learning-based dialogue serviceand that: configures a root dialogue classification node that performs adialogue intent classification task for utterance data input to amachine learning-based automated dialogue system; configures a pluralityof distinct dialogue state classification nodes that are arrangeddownstream of the root dialogue classification node, wherein each of theplurality of distinct dialogue state classification nodes performs adistinct dialogue classification task for a distinct dialogue intentbased on the utterance data; configures a graphical edge connectionbetween the root dialogue classification node and each of the pluralityof distinct state dialogue classification nodes that graphicallyconnects each of the plurality of distinct state dialogue classificationnodes to the root dialogue classification node thereby enabling adialogue path from the root node downstream to each of the plurality ofdistinct state dialogue classification nodes, wherein (i) the rootdialogue classification node, (ii) the plurality of distinctclassification nodes, (iii) and the transition edge connection betweenthe root dialogue classification node and each of the plurality ofdistinct dialogue classification nodes define the graphical dialoguesystem control structure that governs an active dialogue between a userand the machine learning-based automated dialogue system.

In one embodiment, the graphical dialogue system control structurefurther includes: a plurality of distinct dialogue competencysub-networks, wherein each of the plurality of distinct dialoguecompetency sub-networks: separately extends from the root dialogueclassification node via one graphical edge connection, includes aprogenitor node comprising a distinct one of the plurality of distinctstate dialogue classification nodes, wherein one or more sub-dialogueintent nodes are arranged downstream and extend from the progenitornode.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 illustrates a schematic representation of a system 100 inaccordance with one or more embodiments of the present application;

FIG. 1A illustrates a schematic representation of a subsystem of system100 in accordance with one or more embodiments of the presentapplication;

FIG. 2 illustrates an example method in accordance with one or moreembodiments of the present application;

FIG. 3 illustrates a schematic representation of an example controlnetwork for controlling a machine learning-based dialogue system inaccordance with one or more embodiments of the present application; and

FIG. 4 illustrates a schematic representation of a hierarchicalsub-network of state classification nodes for a machine learning-baseddialogue system in accordance with one or more embodiments of thepresent application.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The following description of the preferred embodiments of the presentapplication are not intended to limit the inventions to these preferredembodiments, but rather to enable any person skilled in the art to makeand use these inventions.

Overview

As discussed above, existing virtual assistant implementations do nothave the requisite flexibility to address unrecognized queries orcommands from user in which there are no predetermined rules designedaround narrowly defined dialogue intents. This inflexible structurecannot reasonably and efficiently address the many variances in themanners in which a user may pose a query or command to the virtualassistant.

The embodiments of the present application, however, provide anartificially intelligent machine learning-based dialogue service and/orsystem with natural language processing capabilities that function toprocess and comprehend structured and/or unstructured natural languageinput from a user or input from any other suitable source andcorrespondingly provide highly conversant responses to dialogue inputsto the system. Using one or more trained (deep) machine learning models,such as long short-term memory (LSTM) neural network, the embodiments ofthe present application may function to understand any variety ofnatural language utterance or textual input provided to the system. Theone or more deep machine learning models post deployment can continue totrain using unknown and previously incomprehensible queries or commandsfrom users. As a result, the underlying system that implements the(deep) machine learning models may function to evolve with increasinginteractions with users and training rather than being governed by afixed set of predetermined rules for responding to narrowly definedqueries, as may be accomplished in the current state of the art.

Accordingly, the evolving nature of the artificial intelligence platformdescribed herein therefore enables the artificially intelligent virtualassistant latitude to learn without a need for additional programmingand the capabilities to ingest complex (or uncontemplated) utterancesand text input to provide meaningful and accurate responses.

Additionally, one or more embodiments of the present application enablea configuration of an evolving and dynamic control structure thatfunctions to govern an operation and communication between multiplemachine learning models and dialogue response generating components ofthe machine learning-based dialogue system described throughout thepresent application.

Accordingly, a technical benefit of one or more of these embodimentsinclude an intelligent control of a conversation and/or dialogue betweena third-party and a dialogue agent implemented by the machinelearning-based dialogue system described herein. Specifically, agraphical machine learning-based control network may function to enablea control of prose and content of a response by the machine-learningbased dialogue system separate or independent from controlling a flow ofconversation between a user and a dialogue agent of the dialogue system.

1. System for a Machine Learning-Based Dialogue System

As shown in FIG. 1, a system 100 that automatically trains and/orconfigures machine learning models includes an artificial intelligence(AI) virtual assistant platform 110 (e.g., artificially intelligentdialogue platform), a machine learning configuration interface 120, atraining/configuration data repository 130, a configuration data queue135, and a plurality of external training/configuration data sources140.

Generally, the system 100 functions to implement the artificialintelligence virtual assistant platform 110 to enable intelligent andconversational responses by an artificially intelligent virtualassistant to a user query and/or user command input into the system 100,as described in U.S. patent application Ser. No. 15,797,414, U.S. patentapplication Ser. No. 15,821,010, and U.S. patent application Ser. No.16/668,559, which are all incorporated herein in their entireties bythis reference. Specifically, the system 100 functions to ingest userinput in the form of text or speech into a user interface 160. Atnatural language processing components of the system 100 that mayinclude, at least, the competency classification engine 120 the slotidentification engine 130, and a slot value extractor 135, the system100 functions to identify a competency classification label for the userinput data and parse the user input data into comprehensible slots orsegments that may, in turn, be converted into program-comprehensibleand/or useable features. Leveraging the outputs of the natural languageprocessing components of the system 100, the observables extractor 140may function to generate handlers based on the outcomes of the naturallanguage processing components and further, execute the generatedhandlers to thereby perform various operations that accesses one or moredata sources relevant to the query or command and that also performs oneor more operations (e.g., data filtering, data aggregation, and thelike) to the data accessed from the one or more data sources.

The artificial intelligence virtual assistant platform 110 functions toimplement an artificially intelligent virtual assistant capable ofinteracting and communication with a user. The artificial intelligenceplatform 110 may be implemented via one or more specifically configuredweb or private computing servers (or a distributed computing system;e.g., the cloud) or any suitable system for implementing the system 100and/or the method 200.

In some implementations, the artificial intelligence virtual assistantplatform 110 may be a remote platform implemented over the web (e.g.,using web servers) that is configured to interact with distinct anddisparate service providers. In such implementation, an event such as auser attempting to access one or more services or data from one or moredata sources of the service provider may trigger an implementation ofthe artificially intelligent virtual assistant of the AI platform 110.Thus, the AI virtual assistant platform 110 may work in conjunction withthe service provider to attend to the one or more queries and/orcommands of the users of the service provider. In this implementation,the data sources 160 may be data sources of the service provider thatare external data sources to the AI virtual assistant platform 110.

The competency classification engine 120 together with the slotidentification engine 130 and the slot value extractor 135 preferablyfunction to define a natural language processing (NLP) component of theartificial intelligence platform 110. In one implementation, the naturallanguage processing component may additionally include the automaticspeech recognition unit 105.

The competency classification engine 120 functions to implement one ormore competency classification machine learning models to label userinput data comprising a user query or a user command. The one or morecompetency classification machine learning models may include one ormore deep machine learning algorithms (e.g., a recurrent neural network,etc.) that have been specifically trained to identify and/or classify acompetency label for utterance input and/or textual input. The traininginput used in training the one or more deep machine learning algorithmsof the competency classification engine 120 may include crowdsourceddata obtained from one or more disparate user query or user command datasources and/or platforms (e.g., messaging platforms, etc.). However, itshall be noted that the system 100 may obtain training data from anysuitable external data sources. The one or more deep machine learningalgorithms may additionally be continually trained using user queriesand user commands that were miss-predicted or incorrectly analyzed bythe system 100 including the competency classification engine 120.

The competency classification engine 120 may additionally be configuredto generate or identify one competency classification label for eachuser query and/or user command input into the engine 120. The competencyclassification engine 120 may be configured to identify or select from aplurality of predetermined competency classification labels (e.g.,Income, Balance, Spending, Investment, Location, etc.). Each competencyclassification label available to the competency classification engine120 may define a universe of competency-specific functions available tothe system 100 or the artificially intelligent assistant for handling auser query or user command. That is, once a competency classificationlabel is identified for a user query or user command, the system 100 mayuse the competency classification label to restrict one or morecomputer-executable operations (e.g., handlers) and/or filters that maybe used by system components when generating a response to the userquery or user command. The one or more computer-executable operationsand/or filters associated with each of the plurality of competencyclassifications may be different and distinct and thus, may be used toprocess user queries and/or user commands differently as well as used toprocess user data (e.g., transaction data obtained from external datasources 160).

Additionally, the competency classification machine learning model 120may function to implement a single deep machine learning algorithm thathas been trained to identify multiple competency classification labels.Alternatively, the competency classification machine learning model 120may function to implement an ensemble of deep machine learningalgorithms in which each deep machine learning algorithm of the ensemblefunctions to identify a single competency classification label for userinput data. For example, if the competency classification model 120 iscapable of identifying three distinct competency classification labels,such as Income, Balance, and Spending, then the ensemble of deep machinelearning algorithms may include three distinct deep machine learningalgorithms that classify user input data as Income, Balance, andSpending, respectively. While each of the deep machine learningalgorithms that define the ensemble may individually be configured toidentify a specific competency classification label, the combination ofdeep machine learning algorithms may additionally be configured to worktogether to generate individual competency classification labels. Forexample, if the system receives user input data that is determined to behighly complex (e.g., based on a value or computation of the user inputdata exceeding a complexity threshold), the system 100 may function toselectively implement a subset (e.g., three machine learning algorithmsfrom a total of nine machine learning algorithms or the like) of theensemble of machine learning algorithms to generate a competencyclassification label.

Additionally, the competency classification engine 120 may beimplemented by the one or more computing servers, computer processors,and the like of the artificial intelligence virtual assistance platform110.

The slot identification engine 130 functions to implement one or moremachine learning models to identify slots or meaningful segments of userqueries or user commands and to assign a slot classification label foreach identified slot. The one or more machine learning modelsimplemented by the slot identification engine 130 may implement one ormore trained deep machine learning algorithms (e.g., recurrent neuralnetworks). The one or more deep machine learning algorithms of the slotidentification engine 130 may be trained in any suitable mannerincluding with sample data of user queries and user commands that havebeen slotted and assigned slot values and/or user system derivedexamples. Alternatively, the slot identification engine 130 may functionto implement an ensemble of deep machine learning algorithms in whicheach deep machine learning algorithm of the ensemble functions toidentify distinct slot labels or slot type labels for user input data.For example, slot identification engine 130 may be capable ofidentifying multiple distinct slot classification labels, such asIncome, Account, and Date labels, then the ensemble of deep machinelearning algorithms may include three distinct deep machine learningalgorithms that function to classify segments or tokens of the userinput data as Income, Account, and Date, respectively.

A slot, as referred to herein, generally relates to a defined segment ofuser input data (e.g., user query or user command) that may include oneor more data elements (e.g., terms, values, characters, media, etc.).Accordingly, the slot identification engine 130 may function todecompose a query or command into defined, essential components thatimplicate meaningful information to be used when generating a responseto the user query or command.

A slot label which may also be referred to herein as a slotclassification label may be generated by the one or more slotclassification deep machine learning models of the engine 130. A slotlabel, as referred to herein, generally relates to one of a plurality ofslot labels that generally describes a slot (or the data elements withinthe slot) of a user query or user command. The slot label may define auniverse or set of machine or program-comprehensible objects that may begenerated for the data elements within an identified slot.

Like the competency classification engine 120, the slot identificationengine 120 may implement a single deep machine learning algorithm or anensemble of deep machine learning algorithms. Additionally, the slotidentification engine 130 may be implemented by the one or morecomputing servers, computer processors, and the like of the artificialintelligence virtual assistance platform 110.

The machine learning models and/or the ensemble of machine learningmodels may employ any suitable machine learning including one or moreof: supervised learning (e.g., using logistic regression, using backpropagation neural networks, using random forests, decision trees,etc.), unsupervised learning (e.g., using an Apriori algorithm, usingK-means clustering), semi-supervised learning, reinforcement learning(e.g., using a Q-learning algorithm, using temporal differencelearning), and any other suitable learning style. Each module of theplurality can implement any one or more of: a regression algorithm(e.g., ordinary least squares, logistic regression, stepwise regression,multivariate adaptive regression splines, locally estimated scatterplotsmoothing, etc.), an instance-based method (e.g., k-nearest neighbor,learning vector quantization, self-organizing map, etc.), aregularization method (e.g., ridge regression, least absolute shrinkageand selection operator, elastic net, etc.), a decision tree learningmethod (e.g., classification and regression tree, iterative dichotomiser3, C₄₋₅, chi-squared automatic interaction detection, decision stump,random forest, multivariate adaptive regression splines, gradientboosting machines, etc.), a Bayesian method (e.g., naïve Bayes, averagedone-dependence estimators, Bayesian belief network, etc.), a kernelmethod (e.g., a support vector machine, a radial basis function, alinear discriminate analysis, etc.), a clustering method (e.g., k-meansclustering, expectation maximization, etc.), an associated rule learningalgorithm (e.g., an Apriori algorithm, an Eclat algorithm, etc.), anartificial neural network model (e.g., a Perceptron method, aback-propagation method, a Hopfield network method, a self-organizingmap method, a learning vector quantization method, etc.), a deeplearning algorithm (e.g., a restricted Boltzmann machine, a deep beliefnetwork method, a convolution network method, a stacked auto-encodermethod, etc.), a dimensionality reduction method (e.g., principalcomponent analysis, partial least squares regression, Sammon mapping,multidimensional scaling, projection pursuit, etc.), an ensemble method(e.g., boosting, boostrapped aggregation, AdaBoost, stackedgeneralization, gradient boosting machine method, random forest method,etc.), and any suitable form of machine learning algorithm. Eachprocessing portion of the system 100 can additionally or alternativelyleverage: a probabilistic module, heuristic module, deterministicmodule, or any other suitable module leveraging any other suitablecomputation method, machine learning method or combination thereof.However, any suitable machine learning approach can otherwise beincorporated in the system 100. Further, any suitable model (e.g.,machine learning, non-machine learning, etc.) can be used inimplementing the artificially intelligent virtual assistant and/or othercomponents of the system 100.

The slot value extraction unit 135 functions to generate slot values byextracting each identified slot and assigned slot label of the userquery or user command and converting the data elements (i.e., slot data)within the slot to a machine or program-comprehensible object orinstance (e.g., term or value); that is, the slot label is mapped tocoding or data that a computer or program of the system 100 comprehendsand is able to manipulate or execute processes on. Accordingly, usingthe slot label generated by the slot identification engine 130, the slotextraction unit 135 identifies a set or group of machine orprogram-comprehensible objects or instances that may be applied to slotdata of a slot assigned with the slot label. Thus, the slot extractionunit 135 may convert the slot data of a slot to a machine orprogram-comprehensible object (e.g., slot values) based on the slotlabel and specifically, based on the available objects, instances, orvalues mapped to or made available under the slot label.

The observables extractor 140 functions to use the slot valuescomprising the one or more program-comprehensible objects generated atslot extraction unit 135 to determine or generate one or more handlersor subroutines for handling the data of or responding to the user queryor user command of user input data. The observables extractor 140 mayfunction to use the slot values provided by the slot extraction unit 135to determine one or more data sources relevant to and for addressing theuser query or the user command and determine one or more filters andfunctions or operations to apply to data accessed or collected from theone or more identified data sources. Thus, the coding or mapping of theslot data, performed by slot extraction unit 135, toprogram-comprehensible objects or values may be used to specificallyidentify the data sources and/or the one or more filters and operationsfor processing the data collected from the data sources.

The response generator 150 functions to use the competencyclassification label of the user input data to identify or select onepredetermined response template or one of a plurality of predeterminedresponse templates. For each competency classification label of thesystem 100, the system 100 may have stored a plurality of responsetemplates that may be selected by the response generator 150 based on anidentified competency classification label for user input data.Additionally, or alternatively, the response template may be selectedbased on both the competency classification label and one or moregenerated slot values. In such instance, the one or more slot values mayfunction to narrow the pool of response template selectable by theresponse generator to a subset of a larger pool of response templates toconsider the variations in a query or user command identified in theslot values. The response templates may generally a combination ofpredetermined output language or text and one or more input slots forinterleaving the handler outputs determined by the observables extractor140.

The user interface system 105 may include any type of device orcombination of devices capable of receiving user input data andpresenting a response to the user input data from the artificiallyintelligent virtual assistant. In some embodiments, the user interfacesystem 105 receives user input data in the form of a verbal utteranceand passes the utterance to the automatic speech recognition unit 115 toconvert the utterance into text. The user interface system 105 mayinclude, but are not limited to, mobile computing devices (e.g., mobilephones, tablets, etc.) having a client application of the system 100,desktop computers or laptops implementing a web browser, an automatedteller machine, virtual and/or personal assistant devices (e.g., Alexa,Google Home, Cortana, Jarvis, etc.), chatbots or workbots, etc. Anintelligent personal assistant device (e.g., Alexa, etc.) may be anytype of device capable of touchless interaction with a user toperforming one or more tasks or operations including providing data orinformation and/or controlling one or more other devices (e.g.,computers, other user interfaces, etc.). Thus, an intelligent personalassistant may be used by a user to perform any portions of the methodsdescribed herein, including the steps and processes of method 200,described below. Additionally, a chatbot or a workbot may include anytype of program (e.g., slack bot, etc.) implemented by one or moredevices that may be used to interact with a user using any type of inputmethod (e.g., verbally, textually, etc.). The chatbot or workbot may beembedded or otherwise placed in operable communication and/or control ofa communication node and thus, capable of performing any process or taskincluding, but not limited to, acquiring and providing information andperforming one or more control operations.

As shown in FIG. 1A, a subsystem 170 for intelligently configuring amachine learning-based dialogue system of a subscriber to a machinelearning-based dialogue service includes a dialogue system configurationand management console 175, a dialogue gap assessment module 180, amachine learning training module 185, a dialogue system control module190, a dialogue system deployment engine 195, and a datastore 198.

The dialogue system configuration and management console 175 preferablyenables a remote user and/or administrator of a subscriber or the liketo access, over a network (e.g., the Internet), one or more componentsand/or applications (including programs) of the system 100 for designingand/or configuring one or more aspects of a dialogue system of thesubscriber. In one embodiment, the console 175 may include a subscriber(user) interface, such as a client browser or a client application, thatenables a subscriber to interact with the one or more system orapplication components of the system 100. In some embodiments, thesubscriber interface comprises a programmatic interface that may beimplemented with one or more application programming interfaces thatoperable interact with and/or enable a configuration of any aspect orcomponent of a machine learning-based automated dialogue system or thelike. Additionally, or alternatively, the subscriber interface mayinclude a graphical user interface (e.g., a web-based interface) or thelike.

The dialogue gap assessment module 180 preferably enables a subscriberto perform an assessment of an existing dialogue system of thesubscriber or of an intended dialogue system of the subscriber todetermine gaps in dialogue capabilities and/or dialoguerequirements/needs of the existing or intended dialogue system of thesubscriber, as described in more detail below in the one or more methodsdisclosed herein.

The machine learning training module 185 of the subsystem 170 mayfunction to enable a subscriber to configure one or more machinelearning models for performing one or more dialogue handling-relatedtasks and/or source and configure training data sets for enabling theone or more machine learning models.

The dialogue system control module 190 preferably functions to enable asubscriber to create and/or configure a control structure forcontrolling/implementing a machine learning-based dialogue system of thesubscriber. For instance, in one or more embodiments, the dialoguesystem control module 190 may enable a construction of a dialogue systemcontrol structure, that when executed, causes an implementation of anautomated conversational agent of a dialogue system of a subscriber.

The dialogue system deployment engine 195 preferably functions to accessone or more dialogue system configuration parameters of a dialoguesystem of a subscriber and implement an automated conversational agentalong with the dialogue system of the subscriber. In some embodiments,the dialogue system configuration parameters may include one or more ofa dialogue system control structure of a dialogue system of asubscriber, data sources of a subscriber (e.g., bank account data,patient health records, etc.), business logic for executing computationsand/or user-specific data retrieval and/or transformations, trainingdata corpora, and the like. In one embodiment, the one or more dialoguesystem configuration parameters may be stored in the datastore 198 inassociation with an account for the subscriber. Preferably, thedatastore 198 enables the system 100 to host a plurality of distinctaccounts for a plurality of distinct subscribers to the system/service100.

It shall be noted that any module and/or system component hereinincluding, but not limited to, the dialogue gap assessment module 175,the machine learning training module 180, the dialogue system controlmodule 185, and/or the like may be executed by a distributed one or morecomputers, one or more computing servers, one or more computerprocessors implemented by the system 100. Additionally, oralternatively, any or each of the modules described herein may beexecuted by one or more processors, one or more computers, and/or by adistributed computing system or the like.

2. Method for Intelligently Configuring and Implementing a MachineLearning-Based Control Structure of a Machine Learning-Based DialogueSystem

As shown in FIG. 2, a method 200 for constructing and deploying agraphical machine learning classification control network for anartificially intelligent dialogue system includes configuring a rootnode S210, configuring one or more state nodes S220, configuring aplurality of graphical connections (graphical edges) between nodes S230,deploying the graphical machine learning classification control networkin a live artificially intelligent dialogue system S240. Optionally,configuring logic operations within the graphical machine learningclassification control network S235.

Generally, the method 200 may function to enable a configuration orconstruction of conversational system control flow for a machinelearning-based dialogue system. The conversational system control flowmay include an identification of a plurality of machine learningclassifiers networked and/or mapped together with graphical edges and/ortransitions for executing logic and variable dialogue data. Because themethod 200 enables intelligent construction of a conversation systemcontrol flow, a system (e.g., system 100) implementing the method 200may function to intuitively guide dialogue between a user and a virtualdialogue agent in a highly conversational, uninterrupted, andintelligent manner while resolving an underlying purpose of thedialogue.

It shall be noted that while, in one or more embodiments, in which agraphical user interface (GUI) may be used in construction the dialoguesystem control structure, the graphical edges may have a visuallyperceptible construct within the GUI, in other embodiments, somefeatures of a dialog system control structure, such as the edgeconnections, may be represented and/or enabled using computer code orscripts (e.g., via an API or the like). That is, any feature of thedialogue system control structure that may be constructed and/orassembled using a GUI may also be represented and/or constructed usingan API or other programmatic interface.

S210, which includes configuring a root node, preferably includessetting or positioning a graphical representation of a rootclassification node in a control structure and configuring one or moreoperations of the root classification node within a graphical machinelearning classification control network. The root classification nodepreferably functions as an initial dialogue competency or dialogueintent classifier of conversational data or utterance data obtained froma user interfacing with a dialogue system implementing the method 200.That is, at an outset (e.g., application initialization, etc.) of aconversational interaction between a user and a dialogue systemimplementing the method 200, the root classification node operates toreceive conversational data or input (e.g., verbal input, textual input,etc.) and responsively, processes the initial conversational data orinput to generate one or more competency classifications (labels) or oneor more intent classifications (labels) based on the conversational dataor input and correspondingly, direct the conversation between the userand the machine learning-based dialogue system to a sub-network of stateclassification nodes or a single state classification node (as discussedin more detail below) that is able to handle the conversation based onthe competency classification label.

Additionally, or alternatively, the root classification node may operateto re-classify an ongoing or present conversation between a user and adialogue system implementing the method 200 based on additional orsubsequent conversational data obtained from the user. That is, in someembodiments, S210 may function to use the root classification node togenerate a secondary or subsequent competency classification labelmidstream a conversation between a user and a dialogue agent and/or thelike. In such embodiments, S210 may function to implement the rootclassification node to generate a second distinct competencyclassification label that operates to move a conversation between a userand a dialogue agent from a first sub-network of state nodes associatedwith a first competency classification label to a second sub-network ofstate nodes associated with the second or subsequent competencyclassification label generated by the root classification node.Effectively, a reclassification by the root classification node may betriggered by an inability of a sub-network of state nodes associatedwith an initial competency classification label to handle or generate aproper response to conversational data or otherwise, if a trajectory ofa conversation between the user and a dialogue agent changes towards adifferent competency or topic of conversation. That is, in someembodiments, midstream a conversation between a user and the machinelearning-based dialogue system, S210 may function to determine amisalignment between a dialogue intent competency of a given statedialogue node and a computed dialogue intent of utterance data obtainedfrom the user. In such embodiments, the misalignment in dialogue intentmay be based on a shift or a change in the dialogue intent of the useraway from a prior dialogue intent that matched the current or presentstate dialogue classification node. In another instance, S210 mayfunction to generate a dialogue classification prediction and associatedintent probability match/confidence level that may be below a dialogueintent threshold associated with the present or current state dialogueclassification node. It shall be noted that a reversion to the rootdialogue classification node from a state dialogue classification nodemay be achieved in any suitable manner including, but not limited to,based on a computed misalignment in dialogue intent, an explicit requestby the user (e.g., “start over”, new topic request, etc.), exceeding acapability of a current or present state dialogue classification node,and/or the like. Additionally, it shall be noted that root node mayfunction to support competency and/or intent classification ofconversational input data from any state, including but not limited to,at a completion of a conversation on a topic or competency, a change oftopic and/or a conversation between multiple competencies, and/or thelike.

In a preferred embodiment, one or more competency and/or intentclassification capabilities of the root classification node are enabledusing one or more machine learning classifiers. In one implementation,the root classification node may be implemented by and/or expresslyconfigured with an ensemble of machine learning classifiers capable ofreturning one or more distinct machine learning (competency)classification labels based on (vector) features (e.g., conversationaldata) extracted from conversational input of a user to a dialogue systemimplementing the method 200.

In this implementation, the ensemble of machine learning classifiers mayinclude a plurality of distinct machine learning classifiers working inconcert or independently to generate competency classification labels orpredictions for given conversational data input. For instance, theensemble of machine learning classifiers may include five distinctlytrained machine learning classifiers in which each of the five machinelearning classifiers defining the ensemble may be specifically trainedto classify conversational data for a single competency or intentclassification label and correspondingly, generate the distinctcompetency or intent classification label. It shall be noted that theensemble of machine learning classifiers may include any number ofmachine learning classifiers capable of returning or generating anynumber of machine learning classification labels based on input ofconversational data.

In another implementation, the root classification node may beimplemented by and/or expressly configured with a single combinationalmachine learning classifier that is trained to return or generate one ofa plurality of distinct competency or intent classification labels orpredictions based on features extracted from conversational input of auser to a dialogue system implementing the method 200. That is, thesingle combination machine learning classifier may be specificallytrained to detect all or any type of competency or intent that iscomprehensible by a dialogue system. For example, the singlecombinational machine learning classifier may be trained and/orconfigured to detect five distinct competencies and/or distinct intentsof a user based on input of conversational data. In such example, thesingle combinational machine learning classifier may function to outputa single competency classification label selected from a plurality ofdistinct competency labels for which the single combination machinelearning classifier is trained. Additionally, or alternatively, thesingle combinational machine learning classifier may function to outputmultiple distinct competency classification labels together with aprobability of match of competency or classification intent for each ofthe multiple distinct competency classification labels that is output.

Additionally, S210 may function to configure the root classificationnode to extend to a plurality of distinct state classification nodesand/or sub-networks of state classification nodes. That is, in someembodiments, S210 may function to communicatively connect the rootclassification node to each of a plurality of distinct stateclassification nodes and/or sub-networks of state classification nodes.In this way, the root classification node may function to drive or guidea conversation between a user and a dialogue system implementing themethod 200 by directing conversational input data along a connection ora path to a respective state classification node and/or sub-network ofstate classification nodes based on a competency classification and/orintent classification label generated at the root classification node.That is, the competency classification or intent classification labelproduced at the root classification node may preferably be used togovern a conversational direction along one or more available dialoguecommunication paths within the graphical machine learning classificationcontrol network.

It shall be noted that any suitable and/or state node described hereinmay be additionally or alternatively configured in a similar manner as aroot classification node with single machine learning model with acombination of machine learning models, such as ensemble of machinelearning models, acting in concert to classification data based onconversational input data from a user.

Additionally, or alternatively, the dialogue system control structuremay include a plurality of competency nodes (i.e., state nodes) of whicha subset of the competency nodes may be graphically connected to theroot node. Preferably, each (state/competency) node within the dialogsystem control structure may be configured to perform one or moreclassification tasks and/or one or more inferential tasks based onutterance input and one or more logical operations based on one or morelogical annotations associated with a given node. Each graphical edgebetween two distinct nodes may preferably function to execute a logicfunction and/or variable dialogue data function based on aclassification task, an inferential task, and/or operation of a firstnode of a pair of connected nodes.

S220, which includes configuring state nodes, preferably includesdefining and/or constructing one or more state dialogue classificationnodes in a dialogue system control structure and configuring one or moreoperations of the one or more state classification nodes within agraphical machine learning classification control network.

Preferably, each of the one or more state classification nodes isconfigured to perform a distinct classification task based onconversational data input. That is, in a preferred embodiment, each ofthe one or more state classification nodes may function to implement orbe operated with one or more distinctly trained machine learningclassifier. Accordingly, depending on a conversational flow between auser and a dialogue system implementing the method 200, S220 mayfunction to implement and/or operate one or more of the stateclassification nodes to generate one or more machine learningclassification labels based on conversational data input obtained orderived from a conversation involving a user.

As mentioned previously, the method 200 may include implementing one ormore distinct sub-network of state nodes. Accordingly, in someembodiments, S220 may function to compose a collection and/orcombination of state classification nodes to thereby defined asub-network of state nodes. Preferably, each distinct sub-network ofstate nodes may function to enable conversation within a dialogue systemfor at least one competency. In such preferred embodiments, eachsub-network may include a progenitor node which preferably relates to acoarse competency and/or area of aptitude of a dialogue system. Forinstance, the progenitor node may include a broad category such asfinance.

Each sub-network of state nodes may have a hierarchical structure inwhich the progenitor node is the higher most layer of the hierarchy andsub-state nodes (child nodes) define one or more layers below theprogenitor node, as shown by way of example in FIGS. 3-4. In such ahierarchical architecture, the progenitor node may include a distinctcompetency and/or state dialogue classification node that covers a broador coarse dialogue intent (e.g., Finance dialogue intent, Financecategory, etc.) and each of the sub-state nodes that follow may functionto perform granular dialogue classifications for one distinct dialogueintent (e.g., payments, balances, spending history, etc.) that may becovered by or define part of the coarse dialogue intent.

Preferably, each sub-network includes a single progenitor node that maybe directly and/or operably connected via a graphical edge connection toa root node of a graphical dialogue system control structure.

It shall be noted that the graphical dialog system control structure maybe modified and/or augmented to include any number of distinctsub-networks as a dialog system implementing the method 200 or the likeevolves. For instance, as new broad and/or coarse dialog competenciesare identified by a dialogue system or the like, new sub-networks may becreated and added to an existing graphical dialog system controlstructure thereby evolving the control structure.

In some embodiments, a single state classification node may beimplemented with or used to operate a single machine learningclassifier. In another embodiment, a single state classification nodemay be implemented with or used to operate a plurality and/or anensemble of machine learning classifiers. In an implementation in whicha state classification node is implemented with or used to operate aplurality of distinct machine learning classifiers, the stateclassification node may function to generate a plurality of distinctmachine learning classification labels using the plurality of distinctmachine learning classifiers responsive to conversational data input.

In a preferred embodiment, S220 may function to configure one or moredistinct networks of distinct state classification nodes. In suchembodiment, a single network of state classification nodes preferablyincludes two or more distinct state classification nodes that areconfigured to be in operational communication. The network of stateclassification nodes may function to classify conversation dataaccording to a distinct conversational competency of a dialogue systemimplementing the method 200. Accordingly, S220 may function to configurea plurality of distinct networks of state classification nodes in whicheach of the plurality of distinct state classification nodes isconfigured to perform classification tasks according to one distinctcompetency (e.g., dialogue domain or area of aptitude) of a dialoguesystem implementing the method 200.

Additionally, or alternatively, each distinct network of stateclassification nodes may extend from the root classification node. Thatis, each distinct network of state classification nodes may have adirect communication path from the root classification node. In someembodiments, a distinct network of state classification nodes may beactivated or placed into classification operation based on receiving acompetency classification label, conversational data, and/or anactivation signal from the root classification node via thecommunication path. In such embodiments, other distinct competencyclassification nodes of a dialogue system implementing the method 200may remain in a dormant and/or an inactive state (or deactivated statebased on competency classification prediction). In operation, a dialoguesystem implementing the method 200 preferably functions to operate oractivate a single network of state classification nodes at a time;however, it shall be noted that in the circumstance a conversation flowbetween a dialogue agent or the like of the dialogue system involvesmore than one dialogue competency or dialogue intent, the dialoguesystem may function to operate multiple distinct networks of stateclassification nodes in parallel to enable a dialogue between the userand the dialogue system to shift between distinct networks of thedialogue system control structure.

In a preferred conversation flow, a network (or sub-network) of stateclassification nodes may be placed into operation based on root inputfrom a root classification node. Alternatively, in some embodiment, anetwork of state classification nodes may be placed into operation basedon input from a lateral or distinct network of state classification nodeof a dialogue system implementing the method 200. In such embodiments, afirst network of state classification nodes tasked by the rootclassification node with handling a conversational flow between a userand a dialogue agent may function to pass conversational and/orclassification data laterally to a second distinct network of stateclassification nodes that may be better suited to handle conversationaldata from a user. In such embodiments, the conversation or dialoguebetween the user and the dialogue system may perform a jump or lateralleap (i.e., a transfer) from a first distinct network to a seconddistinct network of the dialogue system control network. That is, adialogue system implementing the method 200 may function to jump or sendan active dialogue between a user and the dialogue system to a seconddistinct network from a first distinct network based on a misalignmentbetween a dialogue intent of an utterance from a user and dialogueintent classification capabilities of the one or more dialogueclassification nodes within a first distinct dialogue classificationsub-network. In a preferred embodiment, when the computed misaligned isdetermined to include a dialogue intent of the user that is within ascope of another sub-network, the dialogue system implementing themethod 200 may function to pass the dialogue handling responsibilitiesfrom a first distinct sub-network having the misalignment to a secondsub-network in which the dialogue intent of the user may be aligned withat least one classification node with the sub-network.

This type of lateral move between two distinct networks may beconsidered a jump or a lateral leap because, in such embodiments, theremay not be a bridge or a graphical transitional edge that explicitlyconnects the first distinct network to the second distinct network.While each of the first distinct network and the second distinct networkmay be impliedly connected with a root dialogue classification node, inone or more embodiments, there may not be a direction connection or adirect path between a state dialogue classification node in the firstdistinct network to a state dialogue classification node in the seconddistinct network.

In a preferred embodiment, each of the dialogue networks of a dialoguesystem may be configured according to one of a plurality of distinctconversational archetypes. For instance, in some embodiments, S220 mayfunction to set or configured each network of state classification nodesas an informational archetype, a confirmational archetype, or anexplorational archetype. Each distinct archetype may be designed orconfigured to achieve a different conversational objective of a dialoguesystem.

As an example, S220 may function to configure a network of stateclassification nodes according to the informational archetype.Accordingly, a conversational objective of the network of stateclassification nodes when configured according to an informationalarchetype may include providing information or other data in aconversational response by a dialogue system responsive to one or morequeries and/or commands from a user. In such embodiments, one or more ofthe state classification nodes that may be configured according to theinformational archetype may be mapped to one or more data sources thatenables the one or more state classification nodes to handle a userquery and/or user command for information.

As another example, S220 may function to configure a network of stateclassification nodes according to the confirmational (executional)archetype. Accordingly, a conversational objective of the network ofstate classification nodes when configured as a confirmational archetypemay include executing one or more actions against an account associatedwith a user or executing some action against some data source based onconversation input from a user. In such embodiments, one or more of thestate classification nodes that may be configured according to theconfirmational archetype may be configured with executional authoritiesthat enables one or more of the state classification nodes to create anaccount and/or execute some action against some data source based onconversational input from a user.

As another example, S220 may function to configure a network of stateclassification nodes according to the explorational (executional)archetype. Accordingly, a conversational objective of the network ofstate classification nodes when configured as an explorational archetypemay include enabling a user to explore a variety of related competenciesand/or topics, preferably associated with a sub-network of nodes) in aneffort to provide a tailored and/or pointed response for resolving auser query, user command, and/or the like associated with conversationalinput from the user.

S230, which includes configuring a plurality of (graphical) connections(graphical edges) between and/or to state nodes, may function toconfigure and/or build operational connections between and/or to stateclassification nodes that define one or more operations or actionscomputed between and/or to state classification nodes in the graphicalmachine learning classification control network. That is, in a preferredimplementation in which the machine learning classification controlnetwork is illustrated with a graphical representation, the plurality ofconnections may each be represented as graphical edges and/or graphicaltransitions between state nodes or a graphical transition that loopsback into a given state node.

Preferably, S230 includes identifying and/or building each of theplurality of connections as one or more of a classification transition,a slot transition, an update transition, and/or the like.

A classification transition (or edge) preferably relates to a transitionbetween at least a pair of state nodes that denotes that a machinelearning classification label was generated by a first state node ororiginating node which may be passed to a second state node as an inputor the like or that enables an active dialogue between a user and thedialogue system to pass to a destination node in the pair of statenodes. The classification transition may typically be represented as agraphical arrow or line extending from a first state node to a secondstate node within the graphical machine learning classification controlnetwork. The graphical representation of the classification transitionmay additionally or alternatively include annotations that describe anoperation of the classification transition and that may function to linka state node to a data corpus, a reference data source, and/or anysuitable computing resource.

A slot transition (or edge) preferably relates to a transition betweenat least a pair of state nodes that denotes that one or more slot valuesmay have been extracted from conversational data generated based on userinput (e.g., a user query, user command, etc.). The slot transition maytypically be represented as a graphical arrow or line extending from afirst state node to a second state node within the graphical machinelearning classification control network. The graphical representation ofthe slot transition may additionally or alternatively includeannotations that describe an operation of the slot transition and thatmay function to link a state node to a data corpus, a reference datasource, and/or any suitable computing resource.

An update transition relates to a specific type of classificationtransition in which the transition connects a pair of state nodes thatare both the same state node and/or indicates a loop of new information,a new classification, or the like back into a same state node. That is,in one or more embodiments, an update transition (or edge) preferablyrelates to a transition that begins at a given state node and revertsback to the state node for purposes of acquiring additional informationfrom a user by the given state node or for indicating that additionalinformation or utterance data acquired from a user interacting with thedialogue system implementing the method 200 may be processed by thegiven state node for a new or an additional classification or forperforming a dialogue action (e.g., generating a response, retrievingdata, executing a requested action by the user, etc.). Accordingly, anupdate transition in some embodiments denotes an action by the dialoguesystem to obtain additional information from a user, which may includeposing a query by a dialogue agent to a user for purposes of collectingadditionally required conversational data (e.g., collecting additionalslot values) for executing some action by the dialogue system. Theupdate transition may typically be represented as a graphical loop thatoriginates and reverts back to a state node within the graphical machinelearning classification control network. The graphical representation ofthe update transition may additionally or alternatively includeannotations that describe an operation of the update transition.

It shall be noted that while S230 may function to configure the one ormore transitions of a machine learning classification control network asone or more a classification transition, a slot transition, and/or anupdate transition, the method 200 may function to implement any suitableand/or type of transition between and/or to state nodes within thecontrol network. For instance, S230 may additionally configure one ormore of the transitions of the control network to include re-classifyingtransitions that operate to revert or move conversational data out of anetwork of state classification nodes to the root classification node inorder to re-classify the conversational data to another competency orthe like. In another example, S230 may function to configure one or moreof the transition of the control network to include multi-hoptransitions which may function to skip one or more state nodes of anordered conversational flow of the control network. That is, in someembodiments, based on conversational data and the one or more operationsperformed at a state node, a multi-hop transition may be triggered thatenables the conversational data and/or conversational flow to skip oneor more (intermediate) downstream nodes to another state node becauseone or more operations of the one or more (intermediate) downstreamnodes may have been satisfied prematurely at an upstream or prior statenode.

Optionally, S235, which includes configuring logic transitions (e.g.,business logic transitions), may function to configure or build one ormore logical transitions between two or more state nodes of the machinelearning classification control network. The one or more logictransitions may include any suitable, business, or other usefulheuristic that enables and/or adds efficiency to a conversational flowbetween a user and a dialogue system implementing the method 200 andenables an execution or completion of one or more business tasks forresponding to conversational input from the user.

In one example, S235 may configure one or more logic transitions and/orlogic annotations that enable a conversational flow to move or jump todisparate (unconnected) sections of a network of state classificationnodes (e.g., some active annotations enable an active dialogue to jumpmultiple nodes), such that even if there is no recognized transitionbetween two state nodes or two distinct networks of state classificationnodes, the logic transition when executed can move the conversationalflow to a second of the two state nodes or the two distinct networks. Inanother example, S235 may configure one or more logic transitions thatenable the control network to interface with one or more externalsystems, external data sources, external resources, and/or the like. Forinstance, the logic transitions may function to execute applicationprogramming interface (API) calls for obtaining data and/or the likefrom an external or remote resource.

S240, deploying the graphical machine learning classification controlnetwork in a live artificially intelligent dialogue system, may functionto implement the graphical machine learning control network as a primaryoperational control structure for implementing a live conversationbetween one or more users and a virtual (digital) dialogue agent of adialogue system implementing the method 200. That is, the graphicalmachine learning classification control network may function to operateas a de facto brain of an artificially intelligent dialogue agent of thedialogue system. For instance, in a preferred embodiment, the dialoguesystem using or implementing the graphical machine learningclassification control network may function to conduct and/or handle oneor more live conversations between real-world users and a virtualdialogue agent of the dialogue system. Accordingly, the graphicalmachine learning classification control network may be used to conduct alive conversation with a user until a disposition, completion, and/ortermination of the conversation between the user and a virtual dialogueagent.

The system and methods of the preferred embodiment and variationsthereof can be embodied and/or implemented at least in part as a machineconfigured to receive a computer-readable medium storingcomputer-readable instructions. The instructions are preferably executedby computer-executable components preferably integrated with the systemand one or more portions of the processors and/or the controllers. Thecomputer-readable medium can be stored on any suitable computer-readablemedia such as RAMs, ROMs, flash memory, EEPROMs, optical devices (CD orDVD), hard drives, floppy drives, or any suitable device. Thecomputer-executable component is preferably a general or applicationspecific processor, but any suitable dedicated hardware orhardware/firmware combination device can alternatively or additionallyexecute the instructions.

Although omitted for conciseness, the preferred embodiments includeevery combination and permutation of the implementations of the systemsand methods described herein.

As a person skilled in the art will recognize from the previous detaileddescription and from the figures and claims, modifications and changescan be made to the preferred embodiments of the invention withoutdeparting from the scope of this invention defined in the followingclaims.

What is claimed:
 1. A method of constructing an automated dialoguecontrol system structure for controlling a machine learning-basedautomated dialogue system, the method comprising: constructing, via auser interface, an automated dialogue system control structure thatcontrols one or more operations of a machine learning-based automateddialogue system, wherein the constructing comprises: constructing afirst graphical component a root dialogue classification node based onuser interface input, wherein the root dialogue classification nodeperforms a dialogue intent classification task for utterance data inputof an active dialogue between a user and the machine learning-basedautomated dialogue system, wherein the root dialogue classification nodepredicts a first dialogue intent classification for the active dialoguebased on the utterance data input; constructing a plurality of secondgraphical components a plurality of distinct dialogue stateclassification nodes downstream of the root dialogue classification nodebased on the user interface input, wherein each of the plurality ofdistinct dialogue state classification nodes performs a distinctdialogue classification task for a distinct dialogue intent based on theutterance data input of the active dialogue; constructing one or moreedge connections between the root dialogue classification node and asubset of the plurality of distinct dialogue state classification nodesthat connects each of the subset of the plurality of distinct dialoguestate classification nodes to the root dialogue classification nodethereby enabling a dialogue path from the root dialogue classificationnode downstream to each of the subset of the plurality of distinctdialogue state classification nodes; annotating one or more distinctexecutable logic instructions, wherein the annotating includes addingexecutable logic instructions at one dialogue state classification nodeof the plurality of distinct dialogue state nodes that, when executed,causes an evaluation of a computed confidence level associated with adialogue intent classification prediction by the one dialogue stateclassification node against a dialogue intent threshold, wherein whenthe computed confidence level does not satisfy the dialogue intentthreshold: (a) the one dialogue state classification node reverts theactive dialogue to the root dialogue classification node, and (b) theroot dialogue classification node predicts a new dialogue intentclassification for the active dialogue based on the utterance datainput; constructing one or more edge connections between the rootdialogue classification node and a subset of the plurality of distinctdialogue state classification nodes that connects each of the subset ofthe plurality of distinct dialogue state classification nodes to theroot dialogue classification node thereby enabling a dialogue path fromthe root dialogue classification node downstream to each of the subsetof the plurality of distinct dialogue state classification nodes,wherein in response to receiving the utterance data, the automateddialogue control structure is executed to control the one or moreoperations of the machine learning-based automated dialogue system. 2.The method according to claim 1, wherein the root dialogueclassification node comprises a single machine learning model that istrained to perform multiple distinct dialogue intent classificationtasks that produces an intent classification prediction based on theinput of the utterance data that aligns with a distinct dialogue intentcompetency of one of the plurality of distinct dialogue stateclassification nodes downstream of the root dialogue classificationnode.
 3. The method according to claim 2, further comprising: passingthe active dialogue between the user and the machine learning-basedautomated dialogue system from the root dialogue classification nodedownstream to a distinct one of the plurality of distinct dialogue stateclassification nodes based on the dialogue intent classificationprediction of the root dialogue classification node.
 4. The methodaccording to claim 1, wherein the root dialogue classification nodecomprises an ensemble of distinct machine learning models, each of thedistinct machine learning models being trained to perform a distinctdialogue intent classification tasks, wherein the ensemble producesmultiple distinct intent classification predictions including a distinctdialogue intent classification prediction for each one of the pluralityof distinct dialogue state classification nodes downstream of the rootdialogue classification node.
 5. The method according to claim 4,further comprising: passing the active dialogue between the user and themachine learning-based automated dialogue system from the root dialogueclassification node downstream to a distinct one of the plurality ofdistinct dialogue state classification nodes based on identifying whichdistinct dialogue intent classification prediction as having a highestlevel of confidence.
 6. The method according to claim 1, wherein the onedialogue state classification node reverts the active dialogue to theroot dialogue classification node based on computing a misalignmentbetween dialogue intent capabilities of the one dialogue stateclassification node and an identified current dialogue intent of theutterance data from the user.
 7. The method according to claim 1,further comprising: a plurality of distinct dialogue competencysub-networks, wherein each of the plurality of distinct dialoguecompetency sub-networks: (i) separately extends from the root dialogueclassification node via one graphical edge connection, (ii) includes aprogenitor node comprising a distinct one of the plurality of distinctdialogue state classification nodes, wherein one or more sub-dialogueintent nodes are arranged downstream and extend from the progenitornode.
 8. The method according to claim 7, wherein each of the pluralityof distinct dialogue competency sub-networks have a hierarchicalstructure in which: the progenitor node defining a top layer of thehierarchical structure is configured with coarse capabilities to performdialogue intent classification tasks for generating predictionsidentifying one or more multiple distinct sub-intents of a singledialogue competency or single dialogue category, the one or moresub-dialogue intent nodes defining one or more subsequent layers fromthe progenitor node is configured with granular capabilities to performdialogue intent classification for at least a distinct one of themultiple distinct sub-intents.
 9. The method according to claim 7,further comprising: when a misalignment between a computed dialogueintent classification of utterance data of a user and a dialogueclassification capability of a sub-dialogue intent classification nodeof a first distinct sub-network of the plurality of distinct dialoguecompetency sub-networks is detected, executing logic instructionsannotated to the first distinct sub-network causing the machinelearning-based automated dialogue system to pass the active dialogue, ina jumping manner, directly from the first distinct sub-network to asecond distinct sub-network that is not connected to the first distinctsub-network with a graphical edge connection having the dialogueclassification capability matching the dialogue intent of the user. 10.The method according to claim 1, further comprising: constructing aclassification transition edge connection, wherein the classificationtransition edge connection relates to a graphical transition connectionbetween at least a pair of distinct dialogue state classification nodesindicating that a machine learning classification prediction has beengenerated by an originating node of the pair that causes the activedialogue between the user and the machine learning-based automateddialogue system to pass from the originating node to a destination nodeof the pair.
 11. The method according to claim 1, further comprising:constructing a slot transition edge connection, wherein the slottransition edge connection relates to a graphical transition connectionbetween at least a pair of distinct dialogue state classification nodesindicating that one or more slot classification values of the activedialogue between the user and the machine learning-based automateddialogue system which are passed pass from an originating node to adestination node of the pair.
 12. The method according to claim 1,further comprising: constructing an update transition loop connection,wherein the update transition loop connection relates to a graphicaltransition connection that graphically originates at a distinct stateclassification node and that loops back into the distinct dialogue stateclassification node indicating an action to be performed by the distinctstate classification node for obtaining additional information beyond aninitial utterance from the user to the distinct state classificationnode.
 13. The method according to claim 1, further comprising:constructing a logic transition edge connection, wherein the logictransition edge connection relates to a graphical logical transitionconnection between at least a pair of distinct dialogue stateclassification nodes that causes an execution or a completion of one ormore business tasks for generating a response to dialogue input from theuser.
 14. The method according to claim 1, further comprising: settingan archetype that is selected from a plurality of distinct archetypesfor each of one or more new dialogue competencies of a target dialoguesystem based on attributes of mishandled utterance data within a targetcorpus of mishandled utterances of one or more distinct corpora ofmishandled utterances.
 15. The method according to claim 1, furthercomprising: deploying the automated dialogue system control structure tocontrol the operation of each of the root dialogue classification nodeand each of the plurality of distinct dialogue state classificationnodes for implementing an artificially intelligent dialogue agent thatinterfaces with the user in the active dialogue.
 16. A system forintelligently configuring a graphical control structure for controllinga machine learning-based automated dialogue system, the systemcomprising: a distributed network of computers that implement a machinelearning-based automated dialogue system; a web-based user interface forconstructing a graphical dialogue system control structure that, whenexecuted, controls one or more operations of the machine learning-basedautomated dialogue service and that: constructs a root dialogueclassification node based on user interface input, wherein the rootdialogue classification node performs a dialogue intent classificationtask for utterance data input of an active dialogue between a user andthe machine learning-based automated dialogue system, wherein the rootdialogue classification node predicts a first dialogue intentclassification for the active dialogue based on the utterance datainput; constructs a plurality of distinct dialogue state classificationnodes that are arranged downstream of the root dialogue classificationnode based on the user interface input, wherein each of the plurality ofdistinct dialogue state classification nodes performs a distinctdialogue classification task for a distinct dialogue intent based on theutterance data input of the active dialogue; annotates one or moredistinct executable logic instructions, wherein the annotating includesadding executable logic instructions at one dialogue stateclassification node of the plurality of distinct dialogue state nodesthat, when executed, causes an evaluation of a computed confidence levelassociated with a dialogue intent classification prediction by the onedialogue state classification node against a dialogue intent threshold,wherein when the computed confidence level does not satisfy the dialogueintent threshold: (a) the one dialogue state classification node revertsthe active dialogue to the root dialogue classification node, and (b)the root dialogue classification node predicts a new dialogue intentclassification for the active dialogue based on the utterance datainput; constructs one or more graphical edge connections between theroot dialogue classification node and a subset of the plurality ofdistinct dialogue state classification nodes that graphically connectseach of the subset of the plurality of distinct dialogue stateclassification nodes to the root dialogue classification node therebyenabling a dialogue path from the root node downstream to each of thesubset of the plurality of distinct dialogue state classification nodes,wherein in response to receiving the utterance data input, thedistributed network of computers executes the graphical dialogue systemcontrol structure that controls the one or more operations of themachine learning-based automated dialogue system.
 17. The systemaccording to claim 16, wherein the graphical dialogue system controlstructure further includes: a plurality of distinct dialogue competencysub-networks, wherein each of the plurality of distinct dialoguecompetency sub-networks: (i) separately extends from the root dialogueclassification node via one graphical edge connection, (ii) includes aprogenitor node comprising a distinct one of the plurality of distinctstate dialogue classification nodes, wherein one or more sub-dialogueintent nodes are arranged downstream and extend from the progenitornode.